Optimization-based Whitening

In natural image understanding, the whitening step plays an important role, especially within many unsupervised feature learning algorithms. Examples of these algorithms include ICA, TICA, Auto-encoder, and so forth. Current whitening techniques include Principal Component Analysis (PCA) and Zero-phase Component Analysis (ZCA). However, the drawbacks of these approaches are obvious: the time complexity is proportional to the cubic of the number of image pixels. When dealing with large images like photos in resolution 640*480, these methods are simply infeasible. In this paper, we probe into ZCA-based whitening filters and observe strong signs of locality, based on which we propose incorporating local receptive fields into whitening to speed up. Secondly, we introduce convolution and symmetry into whitening to further simplify the procedure. Experimental results are reported for both the visual object recognition tasks CIFAR-10 and natural image dataset NIS. Te rest of this paper is organized as following. We introduce ZCA-based whitening in Section 2. Then, we present optimization-based whitening in Section 3. We report experimental results in Section 4 and conclude the paper in Section 5.